You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR adds a new shell script (./scripts/update_py_dependencies.sh) that is used for keeping Python development dependencies up to date. It is not part of the build process, and should be run locally on occasion.
When you run it, it will:
create and activate a temporary virtual env
install the current package dependencies from py/requirements.txt
upgrade the package dependencies to the latest versions available on PyPI
run pip freeze to generate a new py/requirements.txt file
run bazel run //py:requirements.update to generate a new py/requirements_lock.txt file
deactivate and remove the temporary virtual env
This PR also contains updates to the package dependency versions.
🔄 Types of changes
development/packaging tooling
PR Type
Enhancement
Description
Add script to automate Python dependency updates
Upgrade multiple Python package versions in requirements
Regenerate requirements lock file for updated dependencies
Update dependency versions in pyproject.toml and tox.ini
Changes walkthrough 📝
Relevant files
Enhancement
update_py_dependencies.sh
Add script to automate Python dependency updates
scripts/update_py_dependencies.sh
Add new shell script to automate updating Python dependencies
Script creates a temp virtualenv, upgrades packages, regenerates requirements and lock files
The script doesn't handle errors that might occur during package upgrades or when generating new requirements files. Consider adding error checking after critical operations.
echo"upgrading outdated dependencies ..."echo
pip list --outdated |whileread -r line;doif [[ !"${line}"=~"Version Latest"&&!"${line}"=~"----" ]];thenread -ra fields <<<"${line}"echo"upgrading ${fields[0]} from ${fields[1]} to ${fields[2]}"
pip install --upgrade "${fields[0]}==${fields[2]}"> /dev/null
fidone
The script upgrades all outdated dependencies without any filtering mechanism. Consider adding a way to exclude certain packages that might need to be pinned at specific versions.
pip list --outdated |whileread -r line;doif [[ !"${line}"=~"Version Latest"&&!"${line}"=~"----" ]];thenread -ra fields <<<"${line}"echo"upgrading ${fields[0]} from ${fields[1]} to ${fields[2]}"
pip install --upgrade "${fields[0]}==${fields[2]}"> /dev/null
fidone
✅ Fix missing package extraSuggestion Impact:The commit directly implemented the suggestion by changing urllib3==2.4.0 to urllib3[socks]==2.4.0 in requirements.txt, ensuring consistency with pyproject.toml and including the necessary socks dependencies
code diff:
-urllib3==2.4.0+urllib3[socks]==2.4.0
The urllib3 package is specified without the [socks] extra in requirements.txt, but the pyproject.toml file specifies urllib3[socks]. This inconsistency could lead to missing dependencies when installing from requirements.txt directly.
Why: The suggestion identifies a real inconsistency where urllib3[socks] in pyproject.toml differs from urllib3 in requirements.txt, which could lead to missing socks functionality when installing directly from requirements.txt.
Medium
Use robust JSON parsing
The script assumes a specific format for pip list --outdated output, but this format can vary between pip versions. The current implementation may fail if the output columns change or if package names contain spaces. Use a more robust approach to parse the output.
-pip list --outdated | while read -r line; do- if [[ ! "${line}" =~ "Version Latest" && ! "${line}" =~ "----" ]]; then- read -ra fields <<< "${line}"- echo "upgrading ${fields[0]} from ${fields[1]} to ${fields[2]}"- pip install --upgrade "${fields[0]}==${fields[2]}" > /dev/null- fi-done+pip list --outdated --format=json | python -c "+import json, sys+for pkg in json.load(sys.stdin):+ name = pkg['name']+ old_version = pkg['version']+ new_version = pkg['latest_version']+ print(f\"upgrading {name} from {old_version} to {new_version}\")+ import subprocess+ subprocess.check_call(['pip', 'install', '--upgrade', f'{name}=={new_version}'], stdout=subprocess.DEVNULL)+"
Apply / Chat
Suggestion importance[1-10]: 7
__
Why: The suggestion correctly identifies a fragility in parsing pip list --outdated output that could fail with different pip versions or package names containing spaces. Using JSON format provides a more robust solution.
Medium
General
Improve virtual environment cleanup
The script attempts to deactivate the virtual environment directly, which may fail in certain shell contexts. Use a more reliable approach to ensure the virtual environment is properly deactivated before removal.
Why: The suggestion provides defensive programming for virtual environment deactivation, though the issue is relatively minor since the script creates its own temporary environment and should have control over the activation state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
selenium-ci
added
C-py
User description
💥 What does this PR do?
This PR adds a new shell script (
./scripts/update_py_dependencies.sh) that is used for keeping Python development dependencies up to date. It is not part of the build process, and should be run locally on occasion.When you run it, it will:
py/requirements.txtpip freezeto generate a newpy/requirements.txtfilebazel run //py:requirements.updateto generate a newpy/requirements_lock.txtfileThis PR also contains updates to the package dependency versions.
🔄 Types of changes
PR Type
Enhancement
Description
Add script to automate Python dependency updates
Upgrade multiple Python package versions in requirements
Regenerate requirements lock file for updated dependencies
Update dependency versions in pyproject.toml and tox.ini
Changes walkthrough 📝
update_py_dependencies.sh
Add script to automate Python dependency updatesscripts/update_py_dependencies.sh
requirements and lock files
requirements.txt
Update Python dependency versions in requirements.txtpy/requirements.txt
requirements_lock.txt
Regenerate requirements_lock.txt for new dependency versionspy/requirements_lock.txt
pyproject.toml
Update dependency version in pyproject.tomlpy/pyproject.toml
tox.ini
Update test dependency versions in tox.inipy/tox.ini